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ABSTRACT 


wwe presenta formula for ikelthood functionals tor signals in which the corrupting nowe 1s 
modelled ay white noise rather than the usual Wiener process. The main difference is the 
appearance of an additional term corresponding to the conditional mean square error. By 
way of one appleation we Consider the ‘order-disorder’ problem of Shiryayev. 


1. Introduction, The problem of detecting signals in noise is usually 
phrased in the following way: 


yiryp=SU1)4+N (0) O<I1<cT<o (1) 


where ¥(-) is the “observed” stochastic process being the sum of a “signal” 
process S(t), and (ft) the “noise” process, the two processes being mutually 
independent The signal is usually a relauvely “smooth” process in comparison 
with the noise more specifically the noise process has “large” bandwidth 
compared to that of the signal (We use this bandwidth notion in a general way 
since the signal is not assumed to be necessarily stauonary). To allow for this in 
a theoretical way, it was customary in engineering until the 1960's to allow N (1) 
to be “white noise” even though unfortunately there was no precise definition of 
this. Because of this, beginning about 1960, 1t became fashionable to replace the 
model (1) by the “Wiener Process” version: We write 


‘ 
Yin= 7] vs)ds 
J, 
and then make the change 
‘ 
Y(r)= | S(a)do+ W(t), 
ase 


where W (1) 1s a Wiener process. There is no harm in this as long as operations 
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on the observed process are linear as tor example in the case of filtermg theory 
for Gaussian stenals: Simmificant difficulties appear in application as soon as we 
vo to non-lnear operations: as for example in calculating likelihood ratios (for 
continuous-time parameter processes) basically because any non-linear opera- 
Hon is an Ho-integrabin the observed data in which the differenual dY cannot 
simply be replaced by vaade 

Here we shall show that there ty an alternate theory in which this difficulty 
does not arse. In this theory we zo back to (1) but now take N(r) to be 
“white-Gaussian” in the precise sense of nnducing a weak distribution (finitely 
additive measure) on an /y-space. In this cause the process y(-) also induces a 
similiar weak distuibution and we may then calculate the Radon-Nikodym 
denvative of the weak distabuuons. This approach is many ways simpler; 1 
avoids for example the necessity to invoke the Girsanov theorem. But the most 
Huporkant consequence ts a calculable kelthood function. In this way we can 
hyorously justify the somewhat ad-hoc usage of the Wong-Zakai corrections or 
the Stratamovich imtegral or the circle differential of Ito. It turns out that the 
latter are special cases of the more general white noise theory. 

In Secon 2 we indicate the basic nouns and definitions that we need. The 
mam result the kelihood functional formula ts given in Section 3. In parucu- 
lar We see that there is a correction term to the usual Ito formula which can be 
calculated in terms of the conditional mean square error. 

Jn Section 4 we indicate one applicauon of the theory to the so-called 
“disruption! problem imitated by Shiryayev-Kolmogorovy [1]. We show that our 
formula for the conditional probability agrees with the version that has been 
used by Liptser [2]) in actual analogue computer usage. 


2. Basic Notions: White Noise Theory. We shall now indicate briefly the 
relevant nouions from white nome theory, leaving details to {3}. Let 


W(N=L,1(0.0):R,], Onto 


where K, denotes real Fuchdean space of dimension n. By white (Gaussian) 
nome we mean the elements of W (1) under the weak distribution (Gauss 
measure) defined in terms of the characteristic function C(A); 


C(h)=exp— j[AA], 


where |. J, denotes inner-product in W (1). See for example Skorokhod [4]. 
More generally let p denote any weak distribution on W (1). Let f(-) be an 
function mapping W (1) into another Hilbert space H,. Let f(-) be Borel 
measurable. Let ? be any finite dimensional projection on W(1). Then f( Pw), 
where w denotes points in W (1), 1s referred to as a “tame function”, since f (Pw) 
isa random variable in the sense that it induces a countably additive probability 


"Also referred to as “order-disorder”™ problem 


ao eee ee 


= ee eee 


measure on AZ so thatan parteuhu we may speak of the probability that 
f( Po) Borel setin H,. 


We shall say that a Borel measurable function is a weak random variable if for 


any sequence of finite dimensional projections /?. converging strongly to the 


idenuly 
LI( Pw} 
isa Cauchy sequence in probability such that 
C(h)= hunit I [exp I ( Po).h | |. hEH, 


is independent of the particular sequence of projections chosen. In this case the 
weak distibudgon tmduced by /(w) ts by definition that corresponding to C(h). If 
the space MH, as finite dimensional then of course the weak distribution will be 
countably additive so that we will drop the qualifier “weak”. Also if the 
sequence { /(P,0)} is Cauchy actually in the mean of order two, then by the 
Sazonoy theorem we know that C(A) must be the characteristic function of a 
countably additive measure and hence again we may drop the qualifier “weak” 
in this case as well. See [3] for many examples of such random variables. 

Let yy, be Wo weak distributions on HW (1) [or more generally on any 
separable Hilbert space], We shall say that py 1s absolutely continuous with 
respect to ye if given by © +0 we can find 6 +0 such that for any cylinder set C 


with finite-dimensional base: 
y(C )< # whenever 14 (C) <6. 


We shall say that the random variable f(-) 1s the Radon-Nikodym derivative of 
ty with respect to p, if for any cylinder set C and any sequence P, of finite 
dimensional projechons converging strongly to the identity: 


pl j= init f f( Pw) dp, 
c 


We shall be Concerned with this only in the case where p, 1s Gauss measure. 
Finally a bref explanatory word about random variables (taking H, now to 
be finite dimensional), and we take the case of Gauss measure. Let {@,} be any 
orthonormal basis in HW (1). Then we usually take the space of all sequences to 
be @ with fi to be Borel sets therein and determine 2 countably additive 
measure on # which agrees with the given distributions of 4 finite number of the 


variables 


| (5), ws) |ds 
“U 


In the case of Gauss measure, this is equivalent to taking Wiener measure on 


ee —_ 
SL 
. _ 


C [Of We fe dts any random variable, then 
LIC Pw); 


will be a Cauchy sequence converging in probability to some random vartable 
measurable #2 (or ats completion), With white noise however the sample space for 
us as /,. the space of square summiable sequences. In both cases we agree on 
distibutions involving only a finite number of the basic variables: 


[fo ¥), wf ») | ds 


“0 


However not every variable measurable 2 can be expressed as a “random 
Variable” /(w) in our sense. For more on this see [3]. We shall not need to dwell 
on this much in this paper. 


3. The Main Results. Let (2. 8.p) denote a probability triple in the usual 
notabon and S(f,w) be an K, valued jointly measurable stochastic process, 
Os, such that for each 7, O0< 7: 


| E| |S (uw) I)? ]dt< (3.1) 


“0 
Since this implies in particular that 
E| IS Cow)? ] <oo a.e.in/, 
we shall simply assume thruout that 
E| {$ (1,00) ||? | <co forevery1,0<¢. 
Let Wr) denote the Ly-space: 


W'(1)=L,[(O.1]: Ry | 


for each 4, Os ¢. Phen by the Fubsni theorem, 


1 
{ WS (rw)\Pdr< oo ae. inw 
“vU 


and 
o(f,w)=S(-,w), 
where S$(-.w) denotes the function 


S(0,02), OsoK<! 


= = eee 


yields a (Borel) measurable mapping of @ into W(4), where we take the 
o-algebra in HE (7) to be the Borel sets. In particular 


| a 
( (i:h)= bi expe| [ S(a,w),hi(a) | do hew(t) (3.2) 
| “0 


iy. for each ¢, the characteristic function of a countably additive measure on the 


Borel sets of U1). 
Neat let p(s.) Oss <1< % denote white noise elements in W(t), with 


sample points denoted by w’, and let 
V(0,w.w') = $(a,w) + (0,0) USsa<l<o. 


We assume now fand thruout) that the processes $(-,w) and 9(-,w") are 
independent and thus for A(-) in W (1), 


theif Vo. 48)-h(0) || = C(t,A)exp— 7 J, alone, (3.3) 


The first result we need is that the weak distribution induced by (3.3) 1s 
absolutely continuous with respect to that induced by the Gauss measure, and 
that the Radon-Nikodym derivative is a random variable f(7, +), given by: 


jiny= fi exp L{($,8],-2( SA],} ds. (3.4) 


where |, ], denotes inner product in W’(1), and p, the measure corresponding to 
(3.2). We note that we can rewrite this as: 


Sy iee: Ut er 
(uhy= {exp 3 J, Soe)? do 2 f [8 (a.0).h(o) ]do dp. (3.5) 


This result is proved in [5]. 
We wish to show now that (3.5) for each A in W(T) is absolutely continuous 
in [0,7] with derivative given a.e. by (differentiating the integrand): 
4 1Uh)= { 4 LS (wl? 2S (1,0), AU) | exp 
di tee ars ; 


| "Le 2 fe : 
= $(0,w)\?do —2] ¥S(o.w),h(a)d a.e.0<1<T. 
Af fu (0,w)||"do fa (0,w),A(9) 0} « ae 


First of all we note that 


JS oenlPexp ; | f"1S(ayiPae - 2 ['8(0.0),h(9) |e | a 


0 


$ ELS (4)I)7] 


Which as summable m [Oo 7 |. Similarly, 
JI S(.@) ACY Jexp ; | [80.009 do 2f'[S (a.)n(a)] do} 


by Schwartz 


a ye \s (10)\) ACIP 


Which as also summable in [O. 7). Hence 
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us required. 
Newt let us note that, for any € >0, we can find M such that 


du, a\l~e. 
SWC OLS Samy 


Hence 
(ih) >( dnt exp 3 {1S 2( SA],})(l-e) 
=(1 (exp ~} Sup {|S hit) \(exp 5 1mil2) 
(S\|sm 


2(1- ejexp~ S(Al, +m)’. (3.6) 


Hence (for AC-) in WT )) we can take the logarithmic derivative: 


d| {(th) Mh) 
di Og vA f(1h) 


ac.OcreT 
-(f “3 SC)? + [S10 4() J} 
uv 
X exp | [ 8 (0.«))2d0 -2'[8(0,0).A(0) de } de} / 
2 | Yo" ft) ¥ 


exp — } H ‘|S (0,)|l?do — 2 ‘[ $(0,w),h(o0) ] do dp 
2 2\4o 0 
(3.7) 


_ A EB ee 


bet 


Suin=| J SC.werp : | [8 (0,0)\P? do 2f[8(o.hh(oy]do | a) / 
“oO < ) “U J é 


ol} 


[Jer {fis townido 2 f'{8(0.0)."o)] ao} ap] (3.8) 


“2 0 


[ [jis towritern. 1 [is ovonitda— f [$(0,0).4(0)] do | dp} 


aM a ieee anes Sree ne eee eee 


[ few: 11 [1s (o.w01Pdo - ['{St2.e).n(o)] do} «) 


a2 


WS CQA)IP (3.9) 
Then a simple appheation of the Schwartz inequality shows that 
P(th) 20. 


Finally we have then that 


log f(A) pris, l 
og (A) = —-— 
ye J, Sissy. 


a 1 { ek 42 me t 
= - au |S (o,h)| “2f [ s (o,h).(o)]da-+ f'P (o.A)do (3.10) 


Substituting (0,0,w') for A(-) in [3.10], we obtain the “log-likelihood-func- 
tion”. We shall now show that $(1;)) has the interpretation as the conditional 
expectation of S(7,w) given “y(o,0,w'), OSo<72". The latter needs a more 
precise definition which we give now. Let {¢,} be any orthornormal basis in 
W (1) and let P, correspond to the projecuion operator corresponding to the span 


of d),....@,. Let 


i= J .60)9 (0,050) do. 


Let ‘5, denote the sigma-algebra generated by ¢,,...,6,. Let P()» denote 
V(0,w,w'), VG a <7. Then 


E{ S(1,w)|P,P()) |= ELS (1,05, ] 


yields a sequence of tame random variables, a Cauchy sequence in the mean of 


order two, being of Course a martingale sequence (with finite second moment) 
We shall show that this sequence ts equivalent to the sequence 


St Pv) 


so that 


S (iv) El S(1.w)PU)y]. 


bor this first of all, 
ELS IP,P Os] 


is readily seen to be (Bayes Rule) given by: 


{ [Sig Jexp - 3 {UPS -2[ Ps, Pay], } ae, 


“Wit 


{ exp— 3 {iP Si; -2| P,3sPa¥},} 4p, 
Wt) 


This isa Cauchy sequence in the mean of order two. Hence 


[ ( S., Jexp— 3 {Su} -2[ S],) de, 
“Wty 


| exp — {ISH 2[ Sv ],} dy, 

“Wty 

defines a random variable corresponding to the conditional expectation: 
limit E[ [ S$. || Pav | 

More generally, for any @ in W(1), 

hmit E| | Sip || Pay |} 


is the identified with random variable: 


J, [S.g]Jexp— } {SUT ~ 2[ Sv],} dn, 
(4) 


f exp~ }{ Si? -2[ Sv], } ae, 
Wit) 


Now of 
LVS (1,w)- nf" S(0,w)dol|\?-+0 ae. 


t-\l/n 


pe 


and using the fact that 


ECE( SP J) SES? ] 


follows that 


pe 
himth] S (ie) Py | = lim d nf 


mn | 


¥(0.0)dol a 


can be identified with the random variable 
4 -f : ral : 
| | S (weap | | |S (0,w)!\°do 2/ (80.4.0) ]de} 
Jo “0 


[few 31 | 


‘|S (0,0)|)2do 2 {| s(0.0).1(a)] de} ap) 
) “0 


=S(t,y) ae. (3.11) 


In a similar way 


is also a martingale convergent in the mean of order ong, and we can see that 
the limit is identified with the random variable 


f, {Se ]exp— (Suh 2[ 8], da 
- (i) 


fo exp {Si 2[ Sy], } ay 
) 


“Wo 


Again 


; ; beste : 
| piso ~f" nS (0,w)do||? 
t-1/n 
Velisware nS (0,w) do||? 
i-l/a 


QM witha. 


LS Cw) iP If nS (a,w)do 


t= tyr 


Also by Jensen's inequality: 


E\E[SIP,JI< ELIS): 


Hlenee We can readily infer that: 


ELIS (oe)? 1P,y |] 
converzes in the mean of order one, and that + 4) 
SOE po), YI ATF Lp 


-_- 


, 2 ci t bce | L 
ie eee \ j "Sor Pa? ae ee x LSS, ay Yen! A P 


isa random variable 


= E{ |S (4,0)? |2 Cs]. (3.12) 

Moreover we note that substituting )'(0,w, 0’) for h(a) in [3.9], we have: 
P(ty)= ELS (o)PPOy]- iS (ny)IP. 

Now 


E[ YS CoP PCs | (ELS Co PPO |? 


= E[ iS (0) ~ ELS (40)P,Py JPL, By | 
yielding a Cauchy sequence which 1s equivalent to the Cauchy sequence 
P(t; Py). 
Hence we can express P(1,)) also as: 
P(tiy)= ELS (0) ELS (0) P(Oy PRO] 
= #| |S (1.0) - S (ay PIP]. (3.13) 


Finally then we have for the log-likelihood-ratio functional: 


logh(id= ~ sf fis )\\2da - 2 f'[8toy)(0)) do + ['P 0.» do}, 


(3.14) 


which 1s then a generalization of the formula in [5]. 


ee TE ie aa 
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4. Applications. The first application of (3.14) 1s to the case where S (1,0) 
isa Gaussian process becuuse of the resulaing simplification that 


P(t) EY S(t) S (ay)? 
=P(t) 


and as thus mdependent of y(-). Moreover, iis possible, if S$ (1,0) is further an 
lio process, to obtain a differential characterization for P (7) as well as of course 
for S(tzy) See [S}. Tn this paper, however, we shall concentrate on a non-Gaus- 
sian case, studied by Wonham [6] and Liptser-Shiryayev [7]. This 1s the case 
where S(f.w) 1s a (one-dimensional) real-valued process: 


S(1,w)=o0x(t ~ T(w)) 170, 


where r(w) sa Markov time relative to some growing sigma-algebra B (1), and 
xf) is the characterisue function of the positive real-Jine, 120: 


{l 120, 


A=) Gy 1<0, 


and o tsa fixed positive number. Itas readily verified that: 
[ig WE) = | ) =o’P 
E[S (1.0) |=o08( S(.e)|=07Pr[ rw) 20] 


and hence 


“1 
| BE (S (1.0) )dt< ow. 


“0 


The process S(1,) a8 of Course jonmtly measurable nf and w. Let 
y(1,w.w')=axl(1— r(w)) + nw’) O<i<a (4.1) 


where o ~ 0, and n(t4,w’) is White Gaussian noise in WT). Let us calculate the 
log-hkelthood ratio based on [0,7]. Furst let us note that 


E(S(nuy Pts |= oF[ SP ()Y] 


=aS8 (iy) 


2 


=o n(t,y) 20 
So that 


PUvy=o8 (iy) -S$ (ayy 


=o° aliyy— muy) |, (4.2) 


HI 
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ee 
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Where 


Ty) EL x roe PCs |. (4.3) 


Moreover we can readily calculate that lett 2(8) denote the distribution of 


r{w} 


| [exp : o*(t-@)- 20 [sede] ana) 
: 2h ty 
Sane ath aS 


[| "de (0) + {exp : [ot 6) 2o| ‘v(oyds | dP) 
ei oe Ne 4 


alii )= 


The log likehhood ratio can thus be expressed: 


o* [ x(0. do Zo [ zo) Jv(a)do + of (x09) (0. ) do | 


ol “0 v) 
| is if 
| 77 ’ « - 7 ’ l ig . 


where t(2.)) is given by (4.4) 
The simplest case [going back to Wonham [6]] is to take for the distribution 


ol rw) 
Pr(t(w)=O}=P,  Pr(tlwy>r>0l=(-Pjye™, = ADO. 


In this case (4.4) becomes: 


N (ty) 
rlty)= <-——, (4.6) 
(l-p)e “+N(ty) 
where 
Ayn elaae Se visa | ) 
i Mie at a 
1 { a? 6 
+(1- p)d ep) (5 aja of y(s) ds } dé. (4.7) 
Q 


“0 


Ihis as the same formula, except for replacement of ydt by dY, given by 
Shuryayey (If. But the difference appears more strikingly in the “differential 
equation” characterization of 7(/,9) between our version and the “Winer pro- 
cess” version. In our case we can derive an ordinary differential equation for 
nto) by differentiaung directly with respect to rin (4.6), remembering that y'(-) 


(p- 


Co SE SO re 


moan 2 y-funcuon over any tite interval Phis iy done most expeditiously in the 
following way) (simular to the technique in Shiryayey [1]). Rewrite: 


pr) 
a(tyv)= 4 
1+ (1) 
Where 
| \ oa \ 
o(t)= gir) (pra AT gls)dsy, 
i-p / / J's ) J 
where 
a 
(a) =erp ( ‘ ve of y(s)ds 
= 0 
Hence 
u( 1) 
o(1)=A pr) 


Hence using 


a(t) p(t) $ (1) 


a(t) ot) 1+ (1) 
We vel 


2 


m(ty)=(1 n( aS tat m joy (1); n(O.v)=p (4.8) 


which as then the white-noise version of the formula. In contrast to the Wiener 
process version in [1]: 


dz =(lon)(A- o7n?)dt+on(l m)dY (1) (4.9) 


If we now add the Wong-Zakar [8] conection terms to this equation: namely 
subtract 


ad, Qayla- a? )de. 


We obtain 


> 


da =(1 nia ae Jars a m)jodY (1), (4.10) 


Where we note that the only change is replacing y(ayde by d¥ (1) Hence our 
solution is consistent with the Wong-Zakai correction, the Stratanovich mitegral 
{9} and the circle differential formalism of fto [ LO}. Indeed in their calculations 
b in actual simulation of vr). Dashevskn and Liptser [2] also appear to actually 
use fa d}in place of the Mo-version (4.9) that they also derive. They obtain (4.8) 
ina purely formal manner however using the Suatanovich integral to replace the 


flo intexral 
Asa slight (but nevertheless importint in application) generalization, let us 


consider the case 
V(1,0.0') = S$, (1) +X 1(e))S, (+ 0(o’), 


where y€) and rQe) are as before, and the distribution of t(w) 1s the same. We 
can then calculate: . 
(l-p)S\;(je “+ SCN) 


S (ty) = ve 
(l—-p)e-“ + N(t) 


where 


NV (1) =pexp u ['S,(0 do 2['s.ta)r(ardo| 
sortie Sel “0 
‘ 
Pei ve 2 Le 
+(1- p)d [es = | S, (0) do+ [ S;(0)»(o)do ao | 
<9 t) 


bes hs 


Hf}, following the previous example, we now take 


N(t)e™ 
| Oh 
P) 
we have: 
} 
| : S, (1) + 8S, (1)p(1) 
i Sie oa (4.11) 


1+(1) 


Where ot) can be capressed as: 


| 

} 

| 

| Where 


g(t) = exp| ; [S:(o¥'do ie? (0) y(a)do -u] 
\< 40 So 


| if ' 
olt)= g(t) p+(l Af ve(s)ds |, 
t) lip 2) yr Cp} fs ) | 


yf” 


~~ 
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rai frelore 
v(t) 
wo (rps lt) 
ul) 
b (p52 (0 Stor) AJotn 
mo (iy=ar {3 su Aor) tS) (1) (1)p(t) (4.12) 
wath 
P 
o(O) = 
ie a i 


We can then abe vet a differential equation for S (ia), assuming that S$\(1) and 
Sadr) are differentiable, but we omut this deta here, excep to quote the result: 


(dot denoting derivative) 


(Ss th) = SUA) Sy Ch) SCS Cay S,(1)) 


§ (i= = 
Sy) S\ (1) 


+((S)(+ 8 (9)) 


; r « ry of 
A(S,(7- S (ny) - (GS, -Sy( pO) -AY(S (ry) Sy) |)/ 


LSs tt) — 8; CH) (4.13) 
S(OV)=CL pp8, (0) + Sy (ODP. 


Because of the rather complicated mature of this equation, perhaps it would be 
best to use (4 11) above with (4.12) The [to version of (4.12) is given by: 
dep = ACL + ply det Sy Cg t)adY (1). (4.14) 
Finally we note that the statistics of 8 (71) are those of the Cauchy sequence 
S 1, Pay) 
und in turn determined by (4.14) and (4.11). In particular then 


Inf. Stuy) ea 


Fa 


isa Markos time with respect to the growing sizma algebra 
oI 


where (7) 1s the a-algebra generated by {P,v}. hence the statistics are de- 
termimed by the “Wiener process” version. In particular then the optimal 
Stopping ume theory of Shiryayey for the “disorder” problem [1] can be 
exploited, mutatis mutandis 
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